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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments filed August 18, 2004 have been fully considered but they are not 
persuasive. 

2. On page 7, last paragraph, applicant states, " Eleftheriadis is wholly unrelated to a method 
of producing a depth map . . . Eleftheriadis needs to know the location of the objects within the 
video image and in order to locate such objects Eleftheriadis requires a depth map . . . 
Eleftheriadis automatically obtains depth maps from a special "depth camera" 1 1 that produces a 
video signal 101 ... and a depth signal (i.e. depth map) " The examiner disagrees that 
Eleftheriadis is unrelated to a method of producing a depth map but does not dispute that a depth 
map is automatically obtained from a special depth-sensing camera. Claims 1 and 27 
respectively state methods of producing encoding a depth map comprising three or more steps. 
The examiner would first like to point out that the term "depth map" appears only in the 
preambles of claims 1 and 27 and does not appear in any other pending claim except claims 43 
and 44. The claims do not recite a method step of "producing the depth map" that must be 
performed after identifying, allocating an identifying tag, allocating a depth tag, and determining 
and defining an outline. In response to applicant's arguments, the recitations "producing a depth 
map" and "encoding a depth map" have not been given patentable weight because the recitation 
occurs in the preamble. A preamble is generally not accorded any patentable weight where it 
merely recites the purpose of a process or the intended use of a structure, and where the body of 
the claim does not depend on the preamble for completeness but, instead, the process steps or 
structural limitations are able to stand alone. See In re Hirao, 535 F.2d 67, 190 USPQ 15 
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(CCPA 1976) and Kropa v. Robie, 187 F.2d 150, 152, 88 USPQ 478, 481 (CCPA 19.51). The 
examiner would also like to point out that claims 1 and 27 recite that the methods are comprised 
of a list of steps. The word "comprise" is an open-ended statement that does not exclude other 
method steps and there is no limitation that the steps must be performed in a specified order. 
3. On page 9, first paragraph, applicant states, "the action contends that Eleftheriadis 
discloses comparing the 2D image with a library of generic scenes as recited in claim 4. 
However, at col. 7, lines 58-62, Eleftheriadis discloses that past and present video data is used 
"to generate motion vectors". Yet, generation of motion vectors does not represent the use of a 
library of generic scenes." The examiner respectfully disagrees. The term "library of generic 
scenes" was not defined in the claims and has therefore been given the broadest most reasonable 
interpretation. The examiner realizes that column 7, line 58-62 refers to the generation of motion 
vectors using past and present frame video data, but the examiner also points out other pertinent 
sections of the patent. For example, column 16, line 20-43 describes an MPEG-4 encoder 
wherein video object shape information is generated by video object segmentation circuit 1500 
and is provided to a shape coder 1610, while the standard digital video or image signal 101 is 
received and stored in a frame store 1601. The examiner also points out column 18, line 45-65 
wherein in order to account for a temporal component, video object planes (VOPs) are tracked 
from one frame to the next by determining if they belong to the same video object by considering 
both the shape difference and the depth range difference from one frame to the next. In any case, 
the past and present frame video data or the standard digital video or image signal can be 
interpreted as the "library of generic scenes." 
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4. On page 9, second paragraph, applicant states with respect to claim 13, "applicants can 
not find any teaching or suggestion in the cited passage or otherwise that a depth tag is a 
numerical value." The examiner respectfully disagrees. Column 9, line 17-28 shows that an 
array of depth values, or depth map, generated by camera 100 are fed into a 16-bit wide buffer 
512. The examiner realizes that the word "numerical" is not present, but the examiner feels that 
the word "value" and the context of an array or depth map being inputted to a 16-bit buffer 
implies that the values are numerical. Merriam- Webster's Collegiate Dictionary, Tenth Edition, 
defines the word value as 4a: a numerical quantity that is assigned or is determined by 
calculation or measurement. The applicant goes on to state, "the depth signal 102 is not a depth 
tag and thus the fact that the depth signal 102 ranges from 0 to 255 is not pertinent to feature of 
claim 14 " The examiner respectfully disagrees. It is assumed that applicant agrees that the 
depth values referred to with respect to claim 13 satisfy applicant's definition of a depth tag. In 
addition, the term "depth tag" is not further defined in the claims and is therefore given the 
broadest most reasonable interpretation. In column 8, line 62-67 of Eleftheriadis, it is stated that 
an array of depth values is provided via cable 102 to an object segmentation circuit 500. This is 
the same array mentioned with respect to claim 13 and is also the same as the depth signal 
mentioned with respect to claim 14. The examiner feels that there is no difference between a 
"depth tag," a "depth value," and a depth signal. The examiner reiterates that the signal 102, 
which preferably contains a 16-bit linear PCM representation for the depth information, satisfies 
the limitations of claim 14. 

5. On page 9, last paragraph onto page 10, applicant states with respect to claim 18, "Col. 
18, lines 45-55 clearly states that VOPs and not objects are tracked from one frame to another. 
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Eleftheriadis is wholly devoid of any teaching or suggestion of determining and assigning depth 
tags in the portion cited in the action or otherwise " The examiner respectfully disagrees. 
Firstly, the examiner agrees that VOPs are tracked from one frame to the next, but it is also 
determined whether they belong to the same VO (video object) by considering both shape 
difference and depth range difference from one frame to the next, meaning that video objects are 
at least implicitly tracked on successive frames of the image. Secondly, the examiner contends 
that Eleftheriadis does indeed determine and assign depth tags for the at least one object in each 
respective frame since each pixel is assigned a depth in the array of depth values or depth map, 
and regions are selected in the object map to identify which pixels are within each depth range 
(column 17, line 9-20, line 40-52). The depth range difference is then used with the shape 
difference to determine whether VOPs belong to the same VO from one frame to the next. 

6. On page 10, first full paragraph, applicant states, "Independent claim 27 calls for a 
method of encoding a depth map. See above discussion with respect to independent claim 1 . 
Also, for the rejections of claims 5-12, 15-17, 19-23, 29-31, 43 and 44, see above discussion of 
claims 1 and 27. 

7. On page 1 1 , first paragraph, applicant states with respect to claims 43 and 44, "there is no 
suggestion, incentive or motivation in Geshwind , Eleftheriadis or otherwise to apply a depth map 
in converting 2D images into stereoscopic images. Namely, it would not have been obvious to 
modify Geshwind to replace the human operator depth assignment with the application of a 
depth map. Nor would it have been obvious to modify Eleftheriadis for the purpose of 
converting 2D images into stereoscopic images." The examiner respectfully disagrees. 
Geshwind clearly discloses a method to convert two-dimensional motion pictures into left and 
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right (stereoscopic) images for three-dimensional systems using depth information (column 3, 
line 17-37). It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to convert 2D images into stereoscopic (left and right) images by applying a 
depth map generated or encoded according to the methods of claims 1 or 27 as taught by 
Geshwind in order to enhance original two-dimensional images so as to exhibit at least some 
three-dimensional or depth characteristics to be used with three-dimensional exhibition or 
transmission systems (column 2, line 41-54). 

Claim Rejections - 35 USC §102 

8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 1 22(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

9. Claims 1-4, 13, 14, 18, 27, 28, and 32-35 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Eleftheriadis et al. (USPN 6,055,330, previously cited), hereafter Eleftheriadis. 

10. Regarding claim 1, Eleftheriadis discloses a method of producing a depth map (column 9, v 
line 17-28) comprising the steps of: identifying at least one object (via video object selector) 
within a 2D image (column 8, line 52-61; column 17, line 1-20); allocating an identifying tag 
(label, number) to the at least one object (column 10, line 34-45, line 65-column 1 1, line 23; 
column 18, line 36-44); allocating a depth tag (value) to the at least one object (column 10, line 
13-26; column 17, line 40-52); and determining and defining an outline (boundary) of the at least 
one object (column 9, line 45-column 10, line 26; column 17, line 40-52). 
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1 1 . Regarding claim 2, Eleftheriadis discloses that the object outline (contour, border, 
boundary) is defined by a series of coordinates, curves, and/or geometric shapes (column 15, line 
48-61; column 16, line 20-33; column 18, line 56-65). 

12. Regarding claim 3, Eleftheriadis discloses that said identifying tag (label) is a unique 
number (column 18, line 36-44). 

13. Regarding claim 4, Eleftheriadis discloses that identifying said at least one object (object 
map generated) occurs prior to said determining and defining step (scanning clipped histogram to 
find region boundaries; column 17, line 40-52) and includes the step of comparing (inherent 
since a shape difference between two objects is computed) said 2D image with a library of 
generic scenes (stored frames, standard digital image; column 7, line 58-62; column 16, line 20- 
43; column 18, line 45-65). 

14. Regarding claim 13, Eleftheriadis discloses that said depth tag is a numerical value 
(column 9, line 17-28). 

15. Regarding claim 14, Eleftheriadis discloses that said numerical value (depth signal 102) 
ranges from 0 to 255 (included in 16-bit linear representation of values, column 7, line 20-27). 

16. Regarding claim 18, Eleftheriadis discloses tracking the at least one object (video object 
plane, VOP) on successive frames of the image, and determining and assigning depth tags for the 
at least one object in each respective frame (column 18, line 45-55). 

17. Regarding claim 27, Eleftheriadis discloses a method of encoding a depth map (column 
9, line 17-28) including: allocating an object identifier (label) to an object (column 10, line 34- 
45, line 65-column 11, line 23; column 18, line 36-44); allocating a depth tag (value) to said 
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object (column 10, line 13-26; column 17, line 40-52); and defining an outline (boundary) of the 
object (column 9, line 45-column 10, line 26; column 17, line 40-52). 

18. Regarding claim 28, Eleftheriadis discloses that said object outline (contour, border, 
boundary) is defined by a series of x,y (pixel) coordinates, each x,y coordinate being separated 
by a curve (column 15, line 48-61; column 18, line 56-65). 

19. Regarding claim 32, Eleftheriadis discloses that said object outline is defined by at least 
one geometric shape (column 15, line 48-61; column 16, line 20-43). 

20. Regarding claim 33, Eleftheriadis discloses that said at least one geometric shape is 
defined by the form of the shape and the parameters of the shape (column 15, line 48-61; column 
16, line 20-43). 

21 . Regarding claim 34, Eleftheriadis discloses that said allocating the depth tag includes: 
allocating a type (16-bit linear representation) of depth (column 7, line 20-27); and allocating a 
depth for the object (column 10, line 13-26). 

22. Regarding claim 35, Eleftheriadis discloses that the type of depth includes single value, 
linear ramp, or radial ramp (column 7, line 20-27; column 9, line 17-28). 

Claim Rejections - 35 USC § 103 

23. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

24. Claims 5, 19, and 21-23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Eleftheriadis as applied to claims 1 and 2 above. 
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25. Regarding claim 5, Eleftheriadis discloses a boundary computing circuit that identifies 
depth points between objects in the scene (column 9, line 45-50), but does not explicitly disclose 
that determining the outline includes tracing the object pixel by pixel. The examiner takes 
Official Notice that determining the outline of an object by tracing the object pixel by pixel is 
well known in the art. It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to trace the object pixel by pixel as taught by Eleftheriadis in order to 
accurately determine the outline of the object so that it can be extracted or otherwise processed. 

26. Regarding claims 19, 21, and 22, Eleftheriadis discloses that compression is performed in 
both texture and shape individually for each object (column 15, line 56-61), but does not 
explicitly disclose adding a texture bump map to the at least one object, wherein the texture 
bump map is defined by luminance values, chrominance, saturation, color grouping, reflections, 
shadows, focus, and/or sharpness of individual components of the at least one object. The 
examiner takes Official Notice that adding a texture bump map to an object defined by 
luminance, chrominance, saturation, color grouping, reflections, shadows, focus, and/or 
sharpness is well known in the art of computer graphics. It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to add a texture bump map to the at 
least one object in order to give the object a more life-like appearance by adding texture and 
shading to an otherwise flat surface. 

27. Regarding claim 23, Eleftheriadis discloses that depth information signal 102 is a 16-bit 
linear representation of depth (column 7, line 20-27), which means that there are 65,536 possible 
depths in the scene (column 10, line 34-45). It is also disclosed that each macroblock associated 
with an object is a 16 x 16 array of pixels (column 11, line 16-23), which would mean that 
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images of 16x16x16 bit resolution are produced. Eleftheriadis does not disclose that images of 
80x60x8 bit resolution of each 2D image are produced, but the specific dimensions of produced 
images depends on the design and requirements of the particular system. It would have been 
obvious to one of ordinary skill in the art at the time the invention was made to produce 
grayscale images of 80x60x8 bit resolution of each 2D image in order to satisfy specific design 
requirements of the particular system. 

28. Claims 6-10, and 29-31 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Eleftheriadis as applied to claims 1, 2, and 28 above, and further in view of Meek et al. (USPN 
6,029,173, newly cited), hereafter Meek. 

29. Regarding claims 6-8, Eleftheriadis does not explicitly disclose that the step of 
determining the outline further includes using straight lines, curve approximations, or Bezier 
curves to approximate the outline of the at least one object. Meek discloses that the step of 
determining an outline further includes using straight line, curve, and Bezier curve 
approximations to approximate the outline of the at least one object (column 6, line 20-30). It 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
use straight line, curve, and Bezier curve approximations to approximate an outline as taught by 
Meek in order to minimize the storage requirements while providing a high level of accuracy in 
the representation of other-than-straight (curved) features (column 4, line 32-46). 

30. Regarding claim 9, Eleftheriadis does not explicitly disclose that the step of determining 
the outline further includes comparing the object with a library of curves and/or generic 
geometric shapes to approximate the outline. Meek discloses a method and system for 
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representation and use of shape information in geographic databases wherein a step of 
determining an outline (shape) includes comparing (matching) the object with a library of curves 
(column 8, line 50-64) and/or generic or geometric shapes to approximate the outline (column 6, 
line 14-30; column 8, line 20-48). It would have been obvious to one of ordinary skill in the art 
at the time the invention was made to compare the object with a library of curves and/or generic 
geometric shapes as taught by Meek in order to approximate the outline and minimize the 
storage requirements while providing a high level of accuracy in the representation of other-than- 
straight (curved) features (column 4, line 32-46). 

3 1 . Regarding claim 10, Meek discloses that the curve and/or generic or geometric shape are 
scaled to best fit the object (column 8, line 50-64). 

32. Regarding claim 29, Eleftheriadis discloses that each identifying tag is a unique number 
(column 18, line 36-44), but does not disclose that each curve is stored in a library and allocated 
a unique number. Meek discloses that each curve (other-than-straight segment) is stored in a 
library and allocated a unique numerical number (index reference value; column 8, line 60-64). 
It would have been obvious to one of ordinary skill in the art at the time the invention was made 
to store each curve in a library and allocate each curve a unique number as taught by Meek in 
order to further minimize the storage requirements while providing a high level of accuracy in 
the representation of other-than-straight (curved) features (column 4, line 32-46; column 5, line 
6-10). 

33. Regarding claim 30, Meek discloses that said object outline also includes data on the 
orientation (rotation) of each curve (column 8, line 50-64). 
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34. Regarding claim 3 1 , Meek discloses that each said curve is a Bezier curve (column 6, line 
14-30). 

35. Claims 1 1 and 12 are rejected under 35 U.S. C. 103(a) as being unpatentable over 
Eleftheriadis as applied to claim 1 above, and further in view of Nourbakhsh et al. (USPN 
5,793,900, cited on applicant's IDS), hereafter Nourbakhsh. 

36. Regarding claims 1 1 and 12, Eleftheriadis does not disclose that the depth tag includes a 
color code, wherein white represents one of objects relatively close to the viewer, or objects 
relatively distant from the viewer and black represents the other. Nourbakhsh discloses 
generating categorical depth maps using passive defocus sensing wherein a depth map is an array 
of categorical depth values, each value indicating the depth of the scene for a given region such 
that depth values of 2, 1, and 0 correspond to close, medium, and far, respectively (column 5, 
line 9-15). Nourbakhsh further discloses that close regions are lightly shaded, medium regions 
are medium shaded, and far regions are darkly shaded (Figures 2-7; column 5, line 20-25). It 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
represent objects relatively close to the viewer as white, and objects relatively distant from the 
viewer with black as taught by Nourbakhsh in order to give the viewer an impression of depth 
using varying pixel intensities since a brighter portion logically indicates a closer portion which 
is easier to see, and a darker portion indicates a distant portion which is more difficult to see 
(column 5, line 20-31). 
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37. Claims 1 5-17, 20, 43, and 44 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Eleftheriadis as applied to claims 1,19, and 27 above, and further in view of Geshwind et 
al. (USPN 4,925,294, previously cited), hereafter Geshwind. 

38. Regarding claim 1 5, Eleftheriadis discloses a region selector that preprocesses the object 
map to identify which pixels are within each depth range, then identifies one or more video 
objects existing within that region (column 17, line 9-20), but does not disclose that said at least 
one object is further divided into a plurality of segments, each segment being assigned a depth 
tag. Geshwind discloses that said at least one object (image element) is further divided into a 
plurality of segments (portions), each segment being assigned a depth tag (column 5, line 1-4). It 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
divide each object into a plurality of segments and assign each segment a depth tag as taught by 
Geshwind in order to designate areas of objects that are closer to the viewer than other areas of 
the same object (column 5, line 4-15). 

39. Regarding claims 16 and 17, Geshwind discloses that different portions of a single image 
element may be given different depth designations with the computer interpolating depth 
coordinates over the entire element (column 5, line 1-15) and also that depth interpolation may 
be carried out over time between frames and that linear and non-linear interpolation are well 
known (column 5, line 16-27), but does not disclose that the variation in depth is defined by a 
linear or radial ramp function. A ramp is simply a linear, uniformly increasing function. The 
examiner takes Official Notice that linear or radial ramp functions are well known in the art. It 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
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define the variation in depth by a linear or radial ramp function as taught by Geshwind in order 
to create uniformly changing variations in the depth of a single object (column 5, line 1-15). 

40. Regarding claim 20, it has been shown that adding a texture bump map to an object is 
well known in the art (see above discussion of claim 19), but Eleftheriadis does not disclose 
breaking the at least one object into a plurality of components and assigning each component a 
depth tag. Geshwind discloses that said at least one object (image element) is further divided 
into a plurality of segments (portions), each segment being assigned a depth tag (see above 
discussion of claim 15). 

41. Regarding claims 43 and 44, Eleftheriadis does not disclose a method of converting 2D 
images into stereoscopic images applying a depth map generated according to the method of 
claim 1 or applying an encoded depth map generated according to the method of claim 27. 
Geshwind discloses a method to convert two-dimensional motion pictures into left and right 
images for three-dimensional systems using depth information (column 3, line 17-37). It would 
have been obvious to one of ordinary skill in the art at the time the invention was made to 
convert 2D images into stereoscopic (left and right) images by applying a depth map generated 
or encoded according to the methods of claims 1 or 27 as taught by Geshwind in order to 
enhance original two-dimensional images so as to exhibit at least some three-dimensional or 
depth characteristics to be used with three-dimensional exhibition or transmission systems 
(column 2, line 41-54). 

Conclusion 

42. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 . 1 36(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8:30 AM - 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Ryan J. Hesseltine 



